Sparse Representations Are Most Likely to Be the Sparsest Possible

Given a signal S ∈ RN and a full rank matrix D ∈ RN×L with N < L, we define the signal’s overcomplete representations as all α ∈ RL satisfying S = Dα. Among all the possible solutions, we have special interest in the sparsest one – the one minimizing ‖α‖0. Previous work has established that a representation is unique if it is sparse enough, requiring ‖α‖0 < Spark(D)/2. The measure Spark(D) stands for the minimal number of columns from D that are linearly dependent. This bound is tight – examples can be constructed to show that with Spark(D)/2 or more non-zero entries, uniqueness is violated. In this paper we study the behavior of overcomplete representations beyond the above bound. While tight from a worst-case standpoint, a probabilistic point-of-view leads to uniqueness of representations satisfying ‖α‖0 < Spark(D). Furthermore, we show that even beyond this point, uniqueness can still be claimed with high confidence. This new result is important for the study of the average performance of pursuit algorithms – when trying to show an equivalence between the pursuit result and the ideal solution, one must also guarantee that the ideal result is indeed the sparsest.